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ABSTRACT 

We present an automatic road sign detection and recognition service system for automatic 
driving system. Artificial neural networks are models inspired by human nervous system that is 
capable of learning. One of the important applications of artificial neural network is traffic symbol 
Recognition. The traffic symbol system has been studies for many years. It is one of the most 
important research area for enabling vehicle driving assistances. The automatic driving system 
should be simple in order to detect symbol with high responses. The challenge gets more difficult in 
order to make system simple while avoiding complex image processing techniques to detect symbol. 
It uses images taken from the Data base. The propose system consist of three main phases: image 
segmentation, recognition, and classification. 

Keywords — artificial neural networks; object recognition; Open CV; Visual studio; back 
propagation algorithm. 



I. INTRODUCTION 

i. Background 

Complexity of the Recognition Task a normal road in the middle of most cities in the world like the 
one shown for Trowbridge in Figure 1.1, presents a complex scene. It may include trees with different colors, 
and a number of traffic signs to control the traffic on this road. Fundamentally, if a person is asked to point out 
the traffic sign in the image, they can do this easily. 




Figure 1 . 1 : A Road scene near the Trowbridge. 
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However, from the point of view of computer vision, this image contains some difficulties which are addressed 
here: 

• The existence of a number of similar objects (either in color or in shape) in the scene. 

• The presence of obstacles in the scene which can partially or totally occlude the sign. 

• The amount of information in the scene is vast and time is needed to analyze the scene and extract the desired 
information. 



ii. Aims and Objectives of the Research 

The overall aim is to develop a system that can be used for traffic sign inventory. This system can 
assist local or national authorities in the task of maintaining and updating their road and traffic signs by 
automatically detecting and classifying one or more traffic signs from a complex scene (like the one shown in 
Figure 1.1) by segmentation process. This process segmentation process is not our project. This process can be 
done by others. We are going to do our project on segmented image. 




Figure 1.2: an image before segmentation 



Figure 1.3: an image after segmentation 



hi. ARTIFICAL NEURAL NETWORK 

Artificial neural networks have been developed as generalizations of mathematical models of biological 
nervous systems. The basic processing elements of neural networks are called artificial neurons or simply 
neurons or nodes. In a simple mathematical model of neuron, the effects of the synapses are represented by 
connection weights that modulate the effect of associated input signals and the nonlinear characteristics 
exhibited by the neurons that are represented by transfer function. The neuron impulse function is calculated as 
the weighted sum of input signals, with the help of the transfer function. The Learning capability of artificial 
neurons is achieved by adjusting the weights in accordance with chosen learning algorithm. The artificial neural 
network can be trained into two main groups that are supervised and unsupervised learning. In supervised 
learning, the network learns by example with help of some training algorithm and a target value is set, whereas 
in unsupervised learning, no target value is set or no example is given. Unsupervised learning is very complex 
and difficult to implement. 



Inputs 




Output 



Figure 1.4: A neuron in artificial neural network 
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iv. Multi-Layer Artificial Neural Networks and Back Propagation 

To do more complex calculations, such as analyzing the shape of a traffic sign from the input image 
and determining what sign it is, can be performed by adding multiple layers of neurons interconnected. This 
class of networks consists of multiple layers of neurons usually interconnected in a feed-forward way. Each 
neuron in one layer has directed connections to the neurons of the subsequent layer. In many applications the 
units of these networks apply a sigmoid function as an activation function. Multi-layer networks use a variety of 
learning techniques, the most popular being back propagation algorithm, In Back propagation, an abbreviation 
for 'Backward Propagation of Errors', is a method used for training of artificial neural network. From the 
desired or actual output, the network learns from number of inputs, similar to the way a child learns to identify a 
car from group of toys. It is supervised learning method. It requires a dataset of desired output from set of many 
inputs, making up the training set. A back propagation network consists of three layers of units-input layer, 
hidden layer and output layer as shown in figure 2. These units are connected in feed forward manner with input 
units fully connected with units in hidden layer and units in hidden layer fully connected with units in output 
layer. When back propagation network is cycled, an input pattern is propagated forward to the output units 
through input to hidden and hidden to output weights. Back propagation is an iterative process that starts with 
last layer and moves backward through the layers until the first layer is reached. This algorithm is based upon 
Windrows hoff delta learning rule in which the weight adjustment is done through mean square error of output 
response to sample input. The rest of these sample patters are repeatedly presented to the network until the error 
value is minimized. Weights are adjusted according to the error present in the network. 




II. TRAFFIC SIGN RECOGNITION 



Training Images 



Test Image 
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Figure 1.6: System Blocks in Training Mode and Prediction Mode 



In this section, we describe the algorithm for traffic sign recognition. The image processing tool used in 
this paper is a free-and noncommercial Intel® Open Source Computer Vision Library (OpenCV). We used the 
OpenCV library function to do implement various functions of our system. Some of the function we highlighted 
when we felt appropriate in the paper. The traffic sign recognition system we designed consists of a Multi-Layer 
Feed-forward Artificial Neural Network (MLF-ANN) with 3 layers. The first layer is the input layer consisting 
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256 neurons, second layer is the hidden layer consisting 16 neurons and the third layer consists of 6neurons. To 
the 256 neurons will receives 256 binary pixels as input and the MLF-ANN produces 6 outputs for each input. 
Depending on the sign one of the 6 outputs of the network will be predominant which is represents the 
recognized traffic sign. To understand the system we shall understand the working of the designed MLF neural 
network. The MLF neural network operates in two modes: training and prediction mode. For the training of the 
MLF neural network and for the prediction using the MLF neural network we need two data sets, the training set 
and the set that we want to predict (test set). The training mode begins with arbitrary values of the weights -they 
might be random numbers -and proceeds iteratively. Each iteration of the complete training set is called an 
epoch. In each epoch the network adjusts the weights in the direction that reduces the error (see back- 
propagation algorithm). As the iterative process of incremental adjustment continues, the weights gradually 
converge to the locally optimal set of values. Many epochs are usually required before training is completed. 

In order to train and test the neural network we need pool of traffic images. In our study, we collected 
our own images of traffic signs downloaded from internet. We gathered 500 images spanning around six 
different traffic signs. We ensured there is some level of diversity among the images we have collected. All 
these images need to be preprocessed before we give then to the input layer of the network. During the 
prepossessing, Gaussian filter is applied to each image to remove the noise, then the image is binarized and 
cropped to region of interest (ROI) which the area in the image where the traffic sign is placed. The image is 
downscaled to 16x1 6 size. This window of 16x16 pixels are the 256 pixels required as inputs to the ANN. The 
preprocessing flowchart is given in the Figure. 
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Figure 1.7: Preprocessing Flow Chart 
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The preprocessed binary images are later read and given as inputs to the network for training. For 
training we used OpenCV library functions. The OpenCV provides a C++ class called CvANN_MLP to model 
the Artificial Neural Network easily. Below are the functions of this class we used for modeling the network. 

VoidCvANNMLP::create (const Mat&layerSizes, intactivateFunc, doublealpha, doublebeta) 

Where 

LayerSizes: Specifies the number of neurons in each layer 
ActivateFunc: Specifies activation function 

IntCvANN_MLP:: train (const Mat& inputs, const Mat& outputs, const Mat& sample Weights, const 
Mat& sampleldx, CvANN_MLP_TrainParams params, int flags) 
Inputs: Floating-point matrix of input 
Outputs: Floating-point matrix of output 

Sample Weights: Floating-point vector of weights for each sample 

Using the above two functions we have created the ANN model of 256 input neurons, 16 hidden neurons and 6 
output neurons. The train function does the training for the specified number of iterations which we set to 1000. 
The resulting ANN model weights are stored into an XML file. This is output of the system in training mode. 

During the prediction mode, the ANN model in the recognition system shall be able to recognize any 
new test image given as input. To do this the ANN model is recreated with the weights obtained in the training 
mode using CvANN_MLP:: create function. The test image is preprocessed in the same way we have done in 
the training mode. OpenCV provides predict function in the class CvANN_MLP. This function predicts the 
responses of the model to the new input. The preprocessed binary test image is given as input to this function 
which gives six responses of the network for the given input. As we discussed one of the six responses would be 
predominant which is considered as the label for the traffic sign recognized. 

V.RESULTS 



Artificial Neural Networks For Traffic Sign Recognition 
Using OpenCU 




III. CONCLUSION AND FUTURE SCOPE 

The application of neural network in traffic sign recognition has been a field of study recently. Neural 
networks are used to solve the tasks which are difficult for conventional computers or human beings to solve. 
This paper aims at recognition of Traffic sign image recognition using gradient descent based back propagation 
algorithm. The main aim of using this algorithm is to reduce the error which is difference between the 
computed value of neural network and desired value. By using artificial neural network methodology of Traffic 
sign image recognition, 99% accuracy is achieved. The method used in this paper can be further extended to 
recognize multilingual images. 

The application of neural network in traffic sign recognition has been a field of study recently. Neural networks 
are used to solve the tasks which are difficult for conventional computers or human beings to solve. This paper 
aims at recognition of Traffic sign image recognition using gradient descent based back propagation algorithm. 
The main aim of using this algorithm is to reduce the error which is difference between the computed value of 
neural network and desired value. By using artificial neural network methodology of Traffic sign image 
recognition, 99% accuracy is achieved. The method used in this paper can be further extended to recognize 
multilingual images. 
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